Apparatus and method for processing image

ABSTRACT

For automatically laying out a plurality of images, the present invention includes an image input unit which inputs an image; an analysis unit which analyzes the orientation of a principal object and the position of the principal object in the image from the image input by the image input unit; and a layout unit which places the image in accordance with the information analyzed by the analysis unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method for processing an image and arecording medium that stores the method. In particular, the inventionrelates to an image processing method that automatically places aplurality of image data items using the orientation of a principalobject and its position in the image, as well as a recording medium thatstores the method.

2. Description of the Related Art

As a method for using pictures, albums have conventionally been createdby sticking pictures that are printed by developing film onto a mount.

In recent years, creation of albums on digital data becomes possible bydigitizing images, so that users have become able to easily createalbums while changing mount design or layout.

The creation of albums using digital data is easier and more flexiblethan the conventional method of sticking printed paper.

The creation of albums using digital data has become popular as digitalcameras have become widespread, while the number of items ofphotographed image data has increased with increasing memory capacity.

As a result, this produces a new problem in that when users createalbums using digital data, it takes much time to select images from alarge number of images and place them.

To solve these problems, Japanese Patent Laid-Open No. 2006-287917discloses a method for laying out images by specifying the orientationsof objects (main objects), such as persons, in the images using thefaces, bodies, the lines of sight, etc. and performs layout of theimages on the basis of the specified orientations.

Japanese Patent Laid-Open No. 2006-304265 discloses a method for layingout images by calculating the directional components of images fromtheir spatial frequency levels etc. and laying out the images on thebasis of the calculated directional components.

Furthermore, Japanese Patent Laid-Open No. 2007-012030 discloses amethod for laying out images by trimming the images and moving thepositions of the objects so that the main objects in the images do notfall along the fold between pages and a method for placing images bydetermining whether a main object overlaps with another image in thealbum template and if they overlap, trimming the image and moving theposition of the object.

As described in the related arts, although the methods for determininglayout using the orientation of a main object are disclosed, some imageshave a plurality of principal objects.

The related arts do not show a method for determining the orientation inthe case where a plurality of principal objects are present in an image,and there is also an image in which the orientation cannot bedetermined.

To perform balanced layout, it is necessary to consider not only theorientation of the principal object but also the location of theprincipal object. Although there is a technology for performing trimmingdepending on the orientation of the main object, there are many cases inwhich there is no enough area on the background of the principal object,so that the image cannot be trimmed at appropriate location of theprincipal object.

In laying out pictures on an album or the like, providing a space in theorientation of the principal object (including the direction of the lineof sight) allows dynamic and wide reproduction.

Meanwhile, a viewer's point of view tends to be focused on the center ofthe layout mount of an album. Therefore, it is desirable to locate theprincipal object in the center as much as possible.

Accordingly, the principal object should be located at a desirableposition of the final image layout in consideration of not only theorientation of the principal object but also the positional relationshipof the principal object.

SUMMARY OF THE INVENTION

The present invention is made in consideration of the above-describedproblem.

According to an aspect of the present invention, there is provided anautomatic layout apparatus which automatically places a plurality ofimages, the apparatus including an image selection unit which selects animage; an analysis unit which analyzes the orientation of a principalobject specified in the image selected by the image selection unit andthe position of the principal object in the image; and a layout unitwhich determines a position in which the image is to be placed inaccordance with information analyzed by the analysis unit.

In the present invention, in laying out a plurality of image data items,the orientation of a principal object (the face of a person) and theposition thereof in the image are analyzed, and the image data is laidout using the analytical result. This allows appropriate image layout.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an album creating apparatus.

FIG. 2 is an overall flowchart of an album creation application.

FIG. 3 is a diagram of an image selection UI of the album creationapplication.

FIG. 4 is a diagram of layout definition based on analytical results ofimage data.

FIG. 5A is a diagram showing an image in which the orientation of theface is the center and the position of the face in the image is also thecenter.

FIG. 5B is a diagram showing an image in which the orientation of theface is the center and the position of the face in the image is a littleto the left.

FIG. 5C is a diagram showing an image in which the orientation of theface is the center and the position of the face in the image is a littleto the right.

FIG. 6A is a diagram showing an image in which the position of the faceis the center.

FIG. 6B is a diagram showing an image in which the position of the faceis a little to the left.

FIG. 6C is a diagram showing an image in which the position of the faceis a little to the right.

FIG. 7A is a diagram showing an image in which the faces of two personsface the center.

FIG. 7B is a diagram showing an image in which the faces of two personsface leftward.

FIG. 8A is a diagram showing an image in which the faces of two personsface in different directions.

FIG. 8B is a diagram showing an image in which the faces of two personsface in different directions.

FIG. 9A is a diagram showing an image in which there are a plurality ofobjects.

FIG. 9B is a diagram showing an image in which there are a plurality ofobjects.

FIG. 10 is a detailed flowchart of image analysis in FIG. 2.

FIG. 11 is a diagram showing a method for calculating the center ofgravity of a face region, that is, the center of gravity of theprincipal object in the image in the case where there is one principalobject.

FIG. 12 is a diagram showing a method for calculating the centers ofgravity of face regions and the center of gravity of principal objectsin the image in the case where there are two principal objects.

FIG. 13A is a diagram showing a method for calculating the centers ofgravity of face regions and the center of gravity of principal objectsin the image in the case where there are three principal objects.

FIG. 13B shows a method for obtaining the center of gravity of theprincipal objects in the case where there are three principal objects.

FIG. 14 is a diagram showing an example of an album layout.

DESCRIPTION OF THE EMBODIMENTS First Embodiment

FIG. 1 shows a block diagram of an image processing apparatus used inthe present invention, in which it is assumed to use a PC.

A CPU 101 is a central processing unit, which controls the otherfunctional blocks and units. A bridge section 102 provides the functionof controlling data exchange between the CPU 101 and the otherfunctional blocks.

A read only memory (ROM) 103 is a read-only nonvolatile memory, in whicha program called a basic input/output system (BIOS) is stored. The BIOSis a program that is first executed when an image search device isstarted and controls the basic input/output functions of peripheraldevices, such as a secondary storage device 105, a display 107, inputdevices 109, and an output unit 110.

A random access memory (RAM) 104 provides a high-speed readable/writablestorage region.

The secondary storage device 105 is a hard disk drive (HDD) thatprovides a mass storage region. When the BIOS is executed, an operatingsystem (OS) stored in the HDD is executed. The OS provides basicfunctions that can be used in all applications, management of theapplications, and a basic graphical user interface (GUI). Theapplications can provide user interfaces that achieve functions specificto the applications by combining the GUIs provided by the OS.

Data used in the OS and execution programs and jobs of the otherapplications are stored in the RAM 105 or the secondary storage device105 as necessary.

A display control unit 106 performs control to generate the result ofuser's operation performed on the OS or applications as image data ofthe GUI and to display it on the display 107. The display 107 can be aliquid crystal display or a cathode ray tube (CRT) display.

An I/O control unit 108 provides an interface with the plurality ofinput devices 109 and the output unit 110. A typical interface is auniversal serial bus (USB).

Examples of the input devices 109 are a keyboard and a mouse forinputting user's intention to the image search device. Furthermore,image data can be transferred by connection with a memory device, suchas a digital camera, a USB memory, a compact flash (CF) memory, or asecure digital (SD) memory card.

The output unit 110 connects to a printer, thus allowing a desired printresult to be obtained.

FIG. 2 is a flowchart for the operation of an image processing methodaccording to an embodiment of the present invention.

Image data D201 is stored in the storage region of the secondary storagedevice 105 or the storage region of the input device 109, such as a CFmemory or a SD memory card, connected to the I/O control unit 108. Thisimage data D201 is captured using an image-capturing device, such as adigital camera.

In step S201, a user who wants to create an album selects an image groupto be used as image data for the album from the image data D201. As amethod for selection, a method of first displaying the thumbnail imagesof the stored image data D201 and designating desired images therefromis often used.

FIG. 3 shows an example the UI for such image selection. As a UI forimage selection of an album creation application 301, a directory-treedisplay region 302 that presents the storage region of the image datadisplays the configuration of folders stored in the secondary storagedevice 105.

A mouse pointer 306 designates a point position of a mouse, which is oneof the input devices 109. The user can select image data by operatingthe mouse or the keyboard.

The user designates one folder. Image data in the designated folder isdisplayed as thumbnail images in a thumbnail display region 303.

A list of images that the user selects using the mouse pointer 306 isdisplayed in a region 304 as thumbnail images.

In step S202, the user performs the operation to designate thestructures of the individual pages of the album. For example, the userdesignates the maximum number of images, the sticking positions of theindividual images, the sizes, etc. as a layout of one page. All of suchlayout control information may be designated by the user or may beautomatically determined from the number of selected images.

In step S203, the orientation of a principal object in the image dataselected in step S202 and the position in the image are analyzed.

The following description is made assuming that the principal objecthere is the face of a person.

The orientation here means the orientation of the principal object orthe direction of the line of sight.

In this specification, the principal object is the face of a person;alternatively, it is also possible to specify an organic region, such aseyes, to specify the orientation of the face, the direction of the lineof sight, and the position in the image.

Various methods for detecting the principal object are disclosed invarious documents, any of which may be used in this embodiment; forexample, Patent Laid-Open No. 2006-304265, US Publication No.20080285791. For the detection of the face or the organ and thedetermination of the orientation of the face or the direction of theline of sight, conventional methods are disclosed in various documentsand they are not essential part of the present invention, and therefore,detailed description thereof will be omitted here.

In step S203, the image data is analyzed, and the result is stored in ananalytical result D202. In step S204, it is determined whether all theimage data selected in step S201 has been analyzed, and the step S203 isrepeatedly executed until the analysis of all the image data iscompleted. When the analysis of all the image data is completed, theprocess proceeds to step S205, where layout of the images is executed.

FIG. 4 shows layout definition based on the analytical results of imagedata.

If the orientation of the face analyzed in step S203 is the left, theimage is placed on the right of the layout template or on the right-handpage in step S205 irrespective of the position of the face in the image,that is, the position of the principal object. In contrast, if theorientation of the face is the right, the image is placed on the left ofthe layout template or on the left-hand page in step S205 irrespectiveof the position of the face in the image, that is, the position of theprincipal object.

If the orientation of the face analyzed in step S203 is the center, andthe position of the face in the image, that is, the position of theprincipal object, is a little to the left, the image is placed on theright of the layout template or on the right-hand page in step S205. Onthe other hand, if the orientation of the face is the center and theposition of the face in the image, that is, the position of theprincipal object, is a little to the right, the image is placed on theleft of the layout template or on the left-hand page in step S205. Ifthe orientation of the face is the center, and the position of the facein the image, that is, the position of the principal object, is also thecenter, the image is placed at the center of the layout template, ordetermination of the priority level of the placement position is notperformed in step S205.

The position of the principal object in the image can be obtained fromthe position of the center of gravity of the principal object, in theimage, obtained by calculating the center of gravity of the face region,that is, the center of gravity of the principal object, from theposition of the face analyzed in step S203.

FIG. 11 shows a method for calculating the center of gravity of the faceregion, that is, the center of gravity of the principal object in theimage in the case where there is one principal object (one face).

A face region 1200 of an image 1100 in the image data D201 is obtainedby the analysis in step S203. The face region 1200 is obtained as arectangular region, as shown in FIG. 11. The point of intersection ofthe lines extending from the vertexes of the rectangular region isobtained as the center of gravity of the face region 1200, that is, thecenter of gravity G of the principal object.

Reference sign P in FIG. 11 denotes a central region having apredetermined width in the image 1100. If the center of gravity G of theprincipal object is within the central region P, it is determined thatthe principal object is located at the center of the image.

If the center of gravity G of the principal object is at the left of thecentral region P, it is determined that the principal object in theimage is located a little to the left, and if the center of gravity G ofthe principal object is at the right of the central region P, it isdetermined that the principal object in the image is located a little tothe right.

In the image in FIG. 11, the center of gravity G of the principal objectis at the right of the central region P, so that it is determined thatthe principal object in the image is located a little to the right.

The range of the central region P is not limited to that shown in thisembodiment and may be determined depending on the purpose. The methodfor calculating the center of gravity is also not limited to that shownin this embodiment, and a known unit may be used.

FIGS. 5A to 5B show images in which the orientations of the faces arethe center and the positions of the faces (principal objects) in theimages are different.

FIG. 5A shows an image in which the orientation of the face is thecenter and the position of the face in the image is also the center. Inthis case, the placement of the face at the center of the layouttemplate, or determination of the priority level of the placementposition is not performed in step S205.

FIG. 5B shows an image in which the orientation of the face is thecenter and the position of the face in the image is a little to theleft. In this case, the object is placed on the right of the layouttemplate or on the right-hand page in step S205.

FIG. 5C shows an image in which the orientation of the face is thecenter and the position of the face in the image is a little to theright. In this case, the object is placed on the left of the layouttemplate or on the left-hand page in step S205.

FIGS. 6A to 6C show images in which the orientations of the faces arethe left and the positions of the faces in the images are different.

FIG. 6A shows an image in which the position of the face is the center;FIG. 6B shows an image in which the position of the face is a little tothe left; and FIG. 6C shows an image in which the position of the faceis a little to the right. In these cases, all the images in FIGS. 6A to6C are located on the right of the layout template or on the right-handpage.

In the above description, the orientation of the face is described onlyfor the lateral direction; alternatively, an analytical result, such asa vertical direction or a diagonal direction, may be used.

In this case, the layout position is determined from the orientation ofthe face irrespective of the position of the face in the image, exceptwhen the orientation of the face is the center, and the object is placedin the position of the layout template or a page opposite to theorientation of the face.

After the layout is determined in step S205, as described above, layoutinformation D203 for controlling the layout is stored. The layoutinformation D203 includes the number of pages in the album, the names ofimages used, storage destination, the numbers of pages to which theimages are stuck, and stuck positions in the pages. Alternatively, thelayouts of the individual pages in the album may be generated and storedas image data. The storage destination may be either the secondarystorage device 105 or the RAM 104.

In step S206, the result of layout is displayed to the user on thedisplay 107 to determine whether the user is satisfied with the result.If the user is not satisfied, then the process proceeds to step S207,where the user adjusts the layout made in step S205. If the user issatisfied, then the process proceeds to step S208, where the album isprinted and a series of jobs is terminated.

As has been described above, by analyzing the orientation of theprincipal object (the face of the person) in image data and the positionthereof in the image and controlling the layout using the analyticalresult, a good layout can be achieved.

In the above description, the orientation of the face is used as theanalytical result. Alternatively, the above-described process may beperformed using the direction of the line of sight as the analyticalresult.

This embodiment is described using the method of performing layout byanalyzing images that are selected in advance by the user.Alternatively, a method of automatically performing layout by analyzingimage data taken in the image processing apparatus without the need forthe user to select images in advance.

Second Embodiment

In the first embodiment, the method for controlling layout by analyzingthe orientation of a principal object (the face of a person) in imagedata and the position thereof in the image and controlling the layoutusing the analytical result is described.

Although the first embodiment has been described in terms of the casewhere there is one principal object (one face), a case in which thereare two or more principal objects is possible.

In the second embodiment, a method for controlling layout in the casewhere there are two or more principal objects.

Since the method for processing images is the same as that of theforegoing first embodiment, a detailed description thereof will beomitted.

Referring to FIG. 10, a detailed flow of the image analysis in step S203of FIG. 2 will be described.

In step S1001, image data is read from the image data D201, and next instep S1002, all face regions present in the image are detected andanalyzed.

In step S1003, the orientations of the faces are analyzed from theresult of extraction and analysis of the face regions in step S1002, andthe number of faces in predetermined orientations is counted until it isdetermined in step S1004 that the process is completed.

In step S1005, the orientation of the faces is determined from theresult in step S1003. An orientation having the largest count is finallywritten in the analytical result D202 as the orientation of the faces inthe image. If it is not possible to evaluate the counts, that is, in thecase where the orientations of all the faces are different, theorientation of the faces is finally determined to be the center, and iswritten in the analytical result D202.

In step S1006, the positions of the center of gravity are calculatedfrom the face regions detected in step S1002, the positions of theprincipal objects in the image are obtained, and they are written in theanalytical result D202.

FIGS. 7A and 7B show images including two objects who face in the samedirection.

FIG. 7A shows an image in which the faces of the two persons face thecenter. In this case, it is determined in step S203 that the principalobjects face the center. FIG. 7B shows an image in which the faces ofthe two persons face leftward. In this case, it is determined in stepS203 that the principal objects face leftward.

The positions of the principal objects in the image can be obtained bycalculating the centers of gravity of the face regions of the twopersons from the positions of the faces of the two persons analyzed instep S203 and obtaining the center of the gravity of the principalobjects in the image from the centers of the gravity of the faces.

FIG. 12 shows a method for calculating the centers of gravity of theface regions and the center of gravity of the principal objects in theimage in the case where there are two principal objects (two faces).

Face regions 1200 of an image 1100 in the image data D201 are obtainedby the analysis in step S203. The face regions 1200 are obtained asrectangular regions, as shown in FIG. 12. The points of intersection ofthe lines extending from the vertexes of the rectangular regions areobtained as the centers of gravity of the face regions, that is, thecenters of gravity g0 and g1 of the face regions 1200.

Since there are two principal objects (faces) in the image 1100, it isnecessary to obtain the center of gravity G of the principal objects forobtaining the positions of the principal objects.

The center of gravity G of the principal objects can be obtained fromthe midpoint on a line connecting the centers of gravity g0 and g1 ofthe face regions 1200.

Reference sign P in FIG. 12 denotes a central region having apredetermined width. If the center of gravity G of the principal objectsis within the central region P, it is determined that the principalobjects are located in the center of the image 1100.

If the center of gravity G of the principal objects is at the left ofthe central region P, it is determined that the principal objects in theimage are located a little to the left, and if the center of gravity Gof the principal objects is at the right of the central region P, it isdetermined that the principal objects in the image are located a littleto the right of the image.

In the image in FIG. 12, the center of gravity G of the principalobjects is within the central region P, so that it is determined thatthe principal objects in the image are located in the center.

The range of the central region P is not limited to that shown in thisembodiment and may be determined depending on the purpose.

A layout is determined in step S205 on the basis the determination instep S203 in accordance with the definition of the table in FIG. 4.

In the case where there are a plurality of principal objects, thenumbers of the orientations of the individual faces analyzed in stepS203 are counted, and an orientation having the largest count is used asa final processing value.

Accordingly, even if there are two principal objects, if all of themface in the same direction, it can be determined in what direction theprincipal objects face.

FIGS. 8A and 8B show images in which there are two objects who face indifferent directions.

In FIGS. 8A and 8B, the faces of the two persons face in differentdirections. In this case, they are processed in step S203 as images inwhich the principal objects face the center.

The positions of the faces in the images can be obtained from the centerof gravity of the principal objects in the images obtained bycalculating the centers of gravity of the two faces, by a known unit,from the positions of the two faces analyzed in step S203.

A layout is determined in step S205 by the determination in step S203 inaccordance with the definition of the table in FIG. 4.

In the case where there are a plurality of principal objects, thenumbers of the orientations of the individual faces analyzed in stepS203 are counted, and an orientation having the largest count is used asa final processing value.

Accordingly, since there are two principal objects, who face indifferent directions, it is impossible to determine the orientations ofthe faces, and therefore, they are processed as images in which theprincipal objects face the center.

FIGS. 9A and 9B show images in which there are a plurality of objects.

In FIG. 9A, all of the faces of three persons face in differentdirections. In this case, as in the case in FIGS. 8A and 8B, they areprocessed in step S203 as images in which the principal objects face thecenter.

The positions of the principal objects in the images are obtained fromthe center of gravity of the principal objects in the images obtained bycalculating the centers of gravity of the three face regions, by a knownunit, from the positions of the three faces analyzed in step S203.

FIGS. 13A and 13B show a method for calculating the center of gravity ofthe face regions and the center of gravity of the principal objects inthe image in the case where there are three principal objects (faces).

In FIG. 13A, face regions 1200 of an image 1100 in the image data D201are obtained by the analysis in step S203. The face regions 1200 areobtained as rectangular regions, as shown in FIG. 13A. The points ofintersection of the lines extending from the vertexes of the rectangularregions are obtained as the centers of gravity of the face regions, thatis, the centers of gravity ga, gb, and gc of the face regions 1200.

Since there are three principal objects (faces) in the image 1100, it isnecessary to obtain the center of gravity G of the principal objects forobtaining the positions of the principal objects.

FIG. 13B shows a method for obtaining the center of gravity G of theprincipal objects.

Since the centers of gravity ga, gb, and gc of the face regions form arectangle, a known method for obtaining the center of gravity of arectangle can be applied. That is, provided that the midpoints of thethree sides of the triangle ga, gb, and gc are d, e, and f, the point ofintersection of three median lines that connect point ga and point d,point gb and point e, and point gc and point f is determined to be thecenter of gravity G of the principal objects.

Reference sign P in FIG. 13A denotes a central region having apredetermined width. If the center of gravity G of the principal objectsis within the central region P, it is determined that the principalobjects in the image are located in the center of the image.

If the center of gravity G of the principal objects is at the left ofthe central region P, it is determined that the principal objects in theimage are located a little to the left, and if the center of gravity Gof the principal objects is at the right of the central region P, it isdetermined that the principal objects in the image are located a littleto the right.

In the image in FIG. 13A, the center of gravity G of the principalobjects is within the central region P, so that it is determined thatthe principal objects in the image are located at the center.

The range of the central region P is not limited to that of thisembodiment and may be determined depending on the purpose.

A layout is determined in step S205 from the determination in step S203in accordance with the definition of the table in FIG. 4.

In the case where there are a plurality of principal objects, thenumbers of the orientations of the individual faces analyzed in stepS203 are counted, and an orientation having the largest count is used asa final processing value.

Accordingly, since there are three principal objects, who face indifferent directions, it is impossible to determine the orientations ofthe faces, and therefore, they are processed as an image in which theprincipal objects face the center.

In FIG. 9B, the faces of four of five principal objects except one atthe right end face the front (center). In the case where there are aplurality of principal objects, the numbers of the orientations of theindividual faces analyzed in step S203 are counted, and an orientationhaving the largest count is used as a final processing value.

Accordingly, since there are five principal objects, and theorientations of the faces of four persons face the center, theorientation of the faces can be determined as the center having thelargest count.

The positions of the principal objects in the image can be obtained fromthe center of gravity of the principal objects in the image obtained bycalculating the centers of gravity of the five faces, by a known unit,from the positions of the five faces analyzed in step S203.

The method for obtaining the centers of gravity of face regions and thecenter of gravity of principal objects in an image in the case wherethere are three principal objects (faces) has been described above. Evenif the number of principal objects in an image increases, the center ofgravity of face regions and the center of gravity of principal objectsin the image can be obtained by the same method. The center of gravityof the principal objects in FIG. 9B can be obtained by a known methodfor obtaining the center of gravity of a polygon because connecting thecenters of gravity of the face regions forms a pentagon.

As in the above description, the positions of the principal objects inthe image can be obtained from the relationship between thepredetermined central region and the center of gravity of the principalobjects.

A layout is determined in step S205 from the determination in step S203in accordance with the definition of the table in FIG. 4.

As a method for determining the orientation of the faces in the casewhere there are two or more principal objects, the method according tothe orientation of objects having the largest count has been described.However, the present invention is not limited thereto, but it may bedetermined by weighing the orientations of the faces using the sizesthereof.

It is also possible to specify a target person from a plurality ofprincipal objects using a person specifying unit and to determine theorientation of the face thereof.

The method for calculating the positions of the faces in the image fromthe centers of gravity of all the face regions has been described above.Also in this case, the positions of the faces may be calculated byweighing the orientations of the faces using the sizes thereof orspecifying a target person from a plurality of principal objects using aperson specifying unit and determining the orientation of the facethereof.

FIG. 14 shows an example of a layout using the processes in the firstand second embodiments.

Since the image is laid out in consideration of not only theorientations of the principal objects but also the positionalrelationship among the principal objects in the image, the principalobjects can be placed in desired positions.

Other Embodiments

Although the image processing method has been described in terms of asystem for creating an album and printing it, the configuration is notlimited thereto.

For example, the present invention may be used for layout in creating ahome page (HP) on a world wide website (website), such as a photogallery in which image data taken by a digital camera is placed.

It is needless to say that the present invention can be achieved byproviding a system or apparatus with a storage medium (or a recordingmedium) that stores program codes for software that achieves theabove-described functions of the embodiments and by reading the programcodes stored in the storage medium by the system or apparatus.

In this case, the program codes themselves read from the storage mediumachieve the above-described functions of the embodiments, and thestorage medium that stores the program codes constitutes the presentinvention.

It is also needless to say that the above-described functions of theembodiments can be achieved not only by a computer executing the readprogram codes but also by an operating system (OS) that operates on thecomputer performing part or all of the actual processes.

Furthermore, it is needless to say that the above-described functions ofthe embodiments can be achieved when the program codes read from thestorage medium are written to a function expansion card inserted intothe computer or a memory provided in a function expansion unit connectedto the computer, and the function expansion card or a CPU etc. providedin the function expansion unit performs part of all of the actualprocesses.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2008-250528, filed on Sep. 29, 2008, which is hereby incorporated byreference herein in its entirety.

1. An automatic layout apparatus which automatically places a pluralityof images, the apparatus comprising: an image selection unit whichselects an image; an analysis unit which analyzes the orientation of aprincipal object specified in the image selected by the image selectionunit and the position of the principal object in the image; and a layoutunit which determines a position in which the image is to be placed inaccordance with information analyzed by the analysis unit.
 2. Theautomatic layout apparatus according to claim 1, wherein if theorientation of the principal object is the center, the layout unitdetermines a position in which the image is to be placed in accordancewith the result of analysis of the position of the principal object inthe image.
 3. The automatic layout apparatus according to claim 1,wherein if the orientation of the principal object is not the center,the layout unit determines a position in which the image is to be placedin accordance with the result of analysis of the orientation of theprincipal object.
 4. The automatic layout apparatus according to claim1, wherein if there is a plurality of the principal objects, theanalysis unit determines the orientation of principal objects having thelargest count among the plurality of principal objects as an analyticalresult.
 5. The automatic layout apparatus according to claim 1, whereinif there are a plurality of the principal objects and the principalobjects face in different directions, the analysis unit determines thatthe orientation of the principal objects is the center.
 6. The automaticlayout apparatus according to claim 1, further comprising acenter-of-gravity calculation unit which obtains the center of gravityof the principal object, wherein the analysis unit analyzes the positionof the principal object in the image from the center of gravity obtainedfrom the principal object by the center-of-gravity calculation unit. 7.The automatic layout apparatus according to claim 1, further comprisinga center-of-gravity calculation unit which obtains the center of gravityof the principal object, wherein if there are a plurality of theprincipal objects, the analysis unit analyzes the positions of theprincipal objects in the image from a plurality of the centers ofgravity obtained from the plurality of principal objects by thecenter-of-gravity calculation unit.
 8. An automatic layout method forautomatically placing a plurality of images, the method comprising:selecting an image; analyzing the orientation of a principal objectspecified in the image selected in the selecting and the position of theprincipal object in the image; and determining a position in which theimage is to be placed in accordance with information analyzed in theanalyzing.
 9. A computer-readable recording medium that stores acomputer program for operating an automatic layout apparatus whichautomatically places a plurality of images, the computer programcomprising: selecting an image; analyzing the orientation of a principalobject specified in the image selected in the selecting and the positionof the principal object in the image; and determining a position inwhich the image is to be placed in accordance with information analyzedin the analyzing.